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ABSTRACT 


Current education system is changing dynamically to bring out various skills of students/ learners. To make sure the learner learn what they ought to, Outcome based 
education is being adopted in higher education institutions. Educationalists suggest that assessment must be in such a way to evaluate different cognitive level of the 
students as proposed by Bloom. But still some gap prevails on how to set questions to check different cognitive levels of the students. This paper is intended to study 
how the question papers are set, taking “Computer Programming Course” as example and suggest few modifications that may be brought into the question papers to 
assess higher order thinking level of students. Study is made on the questions collected from question papers of different educational institutions and technical 
examinations. Analysis were made on how the questions are distributed based on the Blooms Taxonomy and the keywords used for constructing the questions. 
Findings reveal that the questions are not distributed well on the Blooms Taxonomy. Most of the questions fall in the lower order thinking categories. Higher order 
thinking levels are very much ignored especially in descriptive type questions. Based on the findings few improvements in the question setting were suggested. 
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I. INTRODUCTION 

Outcome based teaching and learning is a pedagogical approach that gives 
importance to the learning outcomes. The learning outcomes are fixed based on 
the experience of the teacher or expert in that field. Curriculum/Syllabus is 
designed in such a way to achieve the required learning outcome [1]. Outcome 
Based Education (OBE), offers a powerful and appealing way of reforming and 
managing engineering education. In the tradition method teaching process was 
considered most important, whereas in Outcome Based Education the product 
that is produced is given importance. It gives importance to the quality of gradu- 
ates produced; the graduates need to demonstrate the required knowledge, skill 
and attitude. Industry needs graduate who are qualified to work from day one. 


Based on the direction given by NBA and other experts, in the recent past, the 
buzz word in all Engineering Institutions and Universities in India is “Outcome 
Based Education and Accreditation” [2]. To ensure the students are gaining 
proper knowledge, skill and attitude, NBA insist/ guide the institutions to prac- 
tice the three learning domain the teaching learning process viz., cognitive, 
psychomotor and affective [3]. 


This paper intend to analyze the question setting patterns, whether it is based on 
the recommendation of Blooms taxonomy and the quality of keywords used in 
the questions. The paper is organized as follows, section II introduces Blooms 
taxonomy and explains the suggestive questions under various revised Blooms 
taxonomy. Section III describes the analysis done on the question collection and 
various patterns observed in question settings. Section IV summarizes the find- 
ings and suggests improvements in the questions settings. 


Il. BLOOMS TAXONOMY 

Cognitive domain of Bloom's Taxonomy[4] is one of the three domains that were 
introduced by Benjamin Bloom in 1950s. This domain is designed to verify a stu- 
dent's cognitive quality during written examination. The famous Bloom's taxon- 
omy consists of six levels i.e. knowledge, comprehension, application, analysis, 
synthesis and evaluation [4]. Later Anderson and Krathwohl [5] revised Bloom's 
taxonomy to fit the more outcome-focused modern education objectives. 
Revised Bloom's taxonomy includes remembering, understanding, applying, 
analyzing, evaluating, and creating. 


The following [6] describe each levels of revised Bloom's Taxonomy: 


Remembering 

Students are expected to exhibit memory of previously learned material by 
recalling facts, terms, basic concepts, and answers. This level serves as the lower 
level or the beginning level of the hierarchy. Questions of this category include 
the facts/syntax they study in the programming language lessons. 


Examples: 
1. List few keywords of C. 
11. Define the syntax of “for” 


Understanding 

Students are expected to understand the concept and able to reproduce in their 
own way. The questions for programming in this category could be translating 
algorithm (e.g.; write output of a program), explaining the processes and flows of 


program and providing examples to illustrate a concept or an algorithm. 


Examples: 
i. Whatis the output of the following code segment? 


ii. Explain the use of “function” with example. 


Applying 

Students are expected to understand the concept and able to apply it ina program 
given. Questions may be given to use the data type, data structure and control 
statements appropriately. 


Examples: 
i. Create an array for storing 100 numbers. 


ii. Write a program to display highest number in the given set of values. 


Analyzing 

Students are able to examine and break information into parts by identifying 
motives or causes. Make inferences and find evidence to support generalizations. 
Students will be able to understand the working of individual component, rela- 
tionships between the statements, how the codes work together and bring the out- 
put expected. 


Examples: 
i. Howthearray can be replaced using structures in the given program? 


ii. Trace the control flow of program given, assuming random input to the vari- 
ables. 


Evaluating 

Students are able to arrive at some conclusion from the given facts, present and 
defend their opinions by making judgments about information, validity of ideas, 
or quality of work based on a set of criteria. 


Example: 
i. Given<<aproblem>> justify the data structure to be used 


il. Evaluate the correctness of a program (for the given problem whether the 
program written addresses it well). 


Creating 
Students are able to integrate the concepts by rearranging components into anew 
whole (a product, plan, pattern or proposal). 


Examples: 
1. Write a program that gets input, marks of 60 students from the user, finds sum 
and mean, and display the values to the user. 


Challenge in classifying questions according to blooms taxonomy is the usage of 
same keywords in different blooms levels [7]. 
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III. QUESTION ANALYSIS 
With so much of work already published, is there any change in the mindset of 
question paper setter? This paper intended to verify the following: 


1. Which cognitive level the questions are targeting? 
2. Whether all the cognitive levels are covered? 


3. What are the keywords used for different levels? whether it matches with the 
question keyword suggested in blooms taxonomy? 


To answer the first question, analysis is done with hundred and three questions 
collected from question banks of various universities and technical exam. Ques- 
tions collection include both descriptive and objective type. The questions col- 
lected are related to “C- Programming”. All the questions were categorized manu- 
ally. Here the questions categorized under “applying” are the questions ask the 
students/learners to write programs for doing simple and direct tasks, the ques- 
tions categorized under “create” are the questions ask the students/learners to 
write programs for bit complicated tasks. Table 1 shows some of the sample ques- 
tions collected. We can observe from the sample questions the quality of objec- 
tive questions 1s better than the quality of descriptive questions, even though the 
cognitive levels tested are same. 


Table 1. Sample question from the course “C-Programming” 


Applying 


Explicit type conversion is known as Understanding 


A] conversion B] disjunction 
C] separation D] casting 


Study the following C program 
void main () { 


What will be the value of the variable a, on the 
execution of the above program 
A] IB] 0 C]-—1 D] None of these 


The meaning of arrow operator in a->b Understanding 
A] ( *a).b B] a.(*b) C] a.b D] None of these 


Analyzing 


@ Remebering 


x = printf(“I See, Sea in C”’); 
printf(‘\n x= % d”, x); } 
A] x= 15 B] x=2 C] Garbage value D] Error 


A] getchar() and putchar() B] scanf() and printf() 
C] input() and output() D] None of these 
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Chart 1. Cognitive level distribution in question collection including both 
descriptive and objective 


Chart 1 shows how the questions are distributed among various bloom's cogni- 
tive level. We could see 58% of the questions fall in the category of “understand- 
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ing’, which falls under the “Lower Order Thinking” category. Even though the 
aim of higher education institutes is to improve higher order thinking, question 
are not very much motivating the students towards higher order thinking. Ques- 
tions asked toward testing the creativity is of only 2%. 


To answer the second question whether the question are distributed in all the lev- 
els, we can observe that the questions from higher order thinking, covering ana- 
lyzing, evaluating, creating is very low and especially the evaluation is not at all 
covered in the questions collection we have taken. To have a closer look two 
other charts Chart 1 and 2 are furnished. 


Chart 2, which describes the distribution of descriptive type question, clearly 
indicates that questions are not asked to verify the “analysis” and “evaluate” 
level in the written examination. In the written examination which is generally a 
long hour duration may include the question to check the “analysis” and “evalu- 
ate” level questions which is very much lagging in the question collection con- 
sidered. 


Cognitive Level Assessment using 
Descriptive Questions 


% 3% 
ox 2% 


Chart 2. Descriptive Questions distribution in various Cognitive 
levels 
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Cognitive Level Assessment using 
Objective Type Questions 
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Chart 3. Objective Type Questions distribution in various Cognitive 
levels 


Chart 3, which describes the distribution of objective type question clearly indi- 
cates that questions are not asked in the “apply” and “create” level in the exami- 
nation since there is no provision for writing program in these technical (multiple 
choice questions) exams. Otherwise question distribution of objective type ques- 
tion among cognitive level is much better compared to the distribution of 
descriptive type questions. 
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Descriptive Questions 
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Chart 4. Question keywords used in descriptive questions 
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Question Keyword Usage in Objective 
Type Questions 


@ What 
@ Which 
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Chart 5. Question keywords used in objective type questions 


The chart 4 and 5 shows the question keywords used in descriptive type and 
objective type questions. In both the types, “what” is most commonly used. 


Remembering 
Match the following — 
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“Which” type questions are present only in objective type questions. Since “Dis- 
cuss”, “Explain” are not used in objective type question as the scope do not per- 
mit it. In the descriptive type questions most of the questions fall under “under- 
stand” category and it uses the keyword “What”, “Explain” and “Discuss”. 
Almost all the questions are asked in the pattern of “Write a programme to do spe- 
cific task” for apply category. The question asked in descriptive type is very 
monotonous. In the following section, paper suggest the possible changes that 
can be brought in to bring more higher order thinking questions, especially in the 
descriptive type questions 


IV. DISCUSSION AND SUGGESTION 

Analysis of question collection shows that 58% of the questions are asked to 
check the understanding of the students and the questions for checking the evalu- 
ation skill is nil. We could see very well that the coverage of all cognitive level is 
missing. Questions for higher order thinking is also very less in the question 
papers. Usage of keyword is also very similar. 


Teachers need to kindle the thinking capacity of the students. To do that the exam- 

ination question paper must have questions of different types using different type 

of keywords. This section suggests few modifications that can be used in the ques- 
tions based on the verbs of blooms digital taxonomy [8 ]. 





Example Questions 





Name the keyword used for reading one single character. 


¢  scanf()-print the character, string, float, integer, octal and hexadecimal values 
¢ printf ()-read character, string, numeric data from keyboard 





¢ putc()-print the character, string, float, integer, octal and hexadecimal values 


Which operator has the lowest priority ? 





A] ++ B] % C] + D] || 


663K 99 


How many times 
int x = 0; 
do { 

printf (““*”’); 
Sr, 
}while (x<10) 


Group the keywords used for file operations 


will be printed? 








Demonstrate the use of “for loop” 








Construct switch — case statements for finding the grade for given mark. 


In the following code make necessary changes to print the statement 5 times 


Analyzing Differentiate the use of arrays and pointers 
What is the output of this program ? 
void main() { 
int a=b=c=10; 
a=b=c=50; 
printf(“\n %od Yod %d’’,a,b,c); } 

A] 50 50 50 B] Compile Time Error 
C]10 10 10 D] Three Gaebage Value 
#include <stdio.h> 
int main () { 

FO 33) 

printf(""This is in loop.\n"); 
j 
return 0;} 





Evaluating 


Which data type/structure can be used to store information of 100 students?Justify. 


For the given problem find whether the code will produce the maximum number? 


/* function returning the maximum between two numbers */ 


int max(int num1, int num2) { 


/* local variable declaration */ 
int result; 


if (num1 > num2) 
result = num 1; 
else 
result = num2; 
return result; 


j 








Creating Draw a flowchart for calculating Electrical bill. 
Develop a C program for real time Bank application. The program is to perform all below operations. 
1. Creating new account — To create a new account 
2. Cash Deposit — To Deposit some amount in newly created account 
3. Cash withdrawal — To Withdraw some amount from your account 
4. Display Account information — It will display all informations of the existing accounts 
5. Log out 
6. Clearing the output screen and display available options 
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Few example questions given in this section will certainly improve the quality of 
question setting in programming language courses. 
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